Skip to main content

MQTT Parsers

To efficiently handle and process data from wireless M-Bus and Metra proprietary meters, we have developed two tools:

  • MQTT Parser Microservice – A Docker-based service that subscribes to raw meter data from an MQTT broker, processes and decrypts messages, and publishes the parsed data for further use.
  • Node-RED MQTT Parser Flow – A low-code visual solution that integrates directly with MQTT, allowing users to parse and handle messages efficiently using Node-RED.

MQTT Parser Microservice​

The MQTT Parser microservice is a Docker-based application designed to parse wireless M-Bus and Metra proprietary meter frames published to an MQTT broker. This service subscribes to acrios/<IMSI>/meter/<meter-id>/raw, processes the data, and publishes the parsed results to acrios/<IMSI>/meter/<meter-id>/parsed.

  • Containerized: Runs in Docker for easy deployment and environment consistency.
  • Configurable: Environment variables allow easy setup of broker details, authentication, and parser selection.
  • Secure: Supports TLS connections via MQTT.
  • Automated: Parses incoming messages from ACR-CV101NI-W-D devices.
  • Encrypted: Supports message decryption using AES keys.

Importing AES Keys​

The MQTT Parser microservice can decrypt messages using AES keys. To import an AES key, create a file named meter_data.csv in the import folder with the following format:

meter_id,driver,aes_key
55124572,minomess,A81A94AB882B83715842AE7AA0B6C9E2
55124573,minomess,A81A94AB882B83715842AE7AA0B6C9E3
55124574,minomess,A81A94AB882B83715842AE7AA0B6C9E4
55124575,minomess,A81A94AB882B83715842AE7AA0B6C9E5
Column NameDescriptionExample
meter_idMeter ID55124572
driverMeter driver or auto for automatic detectionminomess
aes_keyAES keyA81A94AB882B83715842AE7AA0B6C9E2
List of Supported Drivers
DriverMeter Type
autoAutoMeter
abbb23ElectricityMeter
aeriusGasMeter
amiplusElectricityMeter
apator08WaterMeter
apator162WaterMeter
apator172WaterMeter
apatoreitnHeatCostAllocationMeter
apatorna1WaterMeter
aventieshcaHeatCostAllocationMeter
aventieswmWaterMeter
bfw240radioHeatCostAllocationMeter
c5isfHeatMeter
cma12wTempHygroMeter
compact5WaterMeter
dme_07WaterMeter
ebzwmbeElectricityMeter
ehzpElectricityMeter
ei6500SmokeDetector
elfHeatMeter
em24ElectricityMeter
emerlin868WaterMeter
encalHeatMeter
engelmann-fawHeatMeter
esyswmElectricityMeter
eurisiiHeatCostAllocationMeter
ev200WaterMeter
evo868WaterMeter
fhkvdataiiiHeatCostAllocationMeter
fhkvdataivHeatCostAllocationMeter
flowiq2200WaterMeter
gransystemsElectricityMeter
gwfwaterWaterMeter
hcae2HeatCostAllocationMeter
hydrocalm3HeatMeter
hydroclimaHeatCostAllocationMeter
hydrodigitWaterMeter
hydrusWaterMeter
iem3000ElectricityMeter
itronWaterMeter
iwmtx5WaterMeter
izarWaterMeter
kamheatHeatMeter
lansendwDoorWindowDetector
lansenpuPulseCounter
lansenrpRepeater
lansensmSmokeDetector
lansenthTempHygroMeter
lse_07_17WaterMeter
lse_08HeatCostAllocationMeter
microclimaHeatMeter
minomessWaterMeter
mkradio3WaterMeter
mkradio4WaterMeter
mkradio4aWaterMeter
multical21WaterMeter
muniaTempHygroMeter
nemoElectricityMeter
omnipowerElectricityMeter
piigthTempHygroMeter
pollucomfHeatMeter
q400WaterMeter
qcaloricHeatCostAllocationMeter
qheatHeatMeter
qheat_55_usHeatMeter
qsmokeSmokeDetector
qualcosonicHeatCoolingMeter
qwaterWaterMeter
rfmambTempHygroMeter
rfmtx1WaterMeter
sharkyHeatMeter
sharky774HeatMeter
sontex868HeatCostAllocationMeter
supercom587WaterMeter
topaseskrWaterMeter
tsd2SmokeDetector
ultraheatHeatMeter
ultrimisWaterMeter
unismartGasMeter
vario411HeatMeter
vario451WaterMeter
vario451midHeatMeter
waterstarmWaterMeter
watertechWaterMeter
weh_07WaterMeter
wme5HeatMeter
aquastreamWaterMeter builtin
elf2HeatMeter builtin
elsterGasMeter builtin
eltakoElectricityMeter builtin
imeElectricityMeter builtin
iperlWaterMeter builtin
istaheatHeatMeter builtin
itronheatHeatMeter builtin
kampressPressureSensor builtin
relhcaHeatCostAllocationMeter builtin
sensostarHeatMeter builtin
supercalHeatMeter builtin
werhlemodwmWaterMeter builtin
unknownUnknownMeter
caution

Automatic driver detection may misidentify some meters.

info

When data on the MQTT broker changes, the service scans the import folder for an import file. If found, it saves the data to the database, appends timestamps to each record, and renames the file to indicate when the import occurred.

Import File Layout After Database Insertion​

After being imported into the internal database, the meter_data.csv file is renamed to reflect the import timestamp, such as IMPORTED_20241015_084314_meter_data.csv. Additionally, an import_timestamp column is appended to each record.

meter_id,aes_key,import_timestamp
123456789,F8B24F12F9D113F680BEE765FDE67EC0,2025-02-05 13:33:34
987654321,ABCDE12345ABCDE12345ABCDE12345AB,2025-02-05 13:33:34
1403327,ABCDE12345ABCDE12345ABCDE12345AB,2025-02-05 13:33:34

Running the MQTT Parser Microservice​

info

The MQTT Parser microservice can use either the internal wM-Bus parser or the ACRIOS Systems' backend for processing data. Set ENABLE_INTERNAL_WMBUSMETERS_PARSER to true to use the internal parser, or false to rely on the remote backend.

caution

The Metra parser API is not publicly available! Contact ACRIOS Systems to get bearer token for the API.

  1. Clone the MQTT Parser repository

    git clone https://sw.acrios.com/acrios/mqtt-parser.git

  2. Navigate to the cloned repository and modify the docker-compose.yml for your MQTT broker:

    version: '3'
    services:
    mongo:
        image: mongo:4.4.18
        container_name: mongo_db
        ports:
        - "27017:27017"
        volumes:
        - ./db:/data/db
        networks:
        - mqtt-net

    wmbusmeters:
        build:
        context: ./wmbusmeters
        dockerfile: ./Dockerfile
        container_name: wmbusmeters_container
        networks:
        - mqtt-net

    mqtt-parser:
        build: .
        container_name: mqtt_parser
        environment:
        - PYTHONUNBUFFERED=1
        - BROKER=test.mosquitto.org
        - PORT=1883
        - USERNAME=
        - PASSWORD=
        - TOPIC_PREFIX=acrios/+/meter/+/raw
        - ENABLE_INTERNAL_WMBUSMETERS_PARSER=true
        - API_WMBUSMETERS=https://backend.wmbus.testvps.acrios.com/parser/wmbusmeters/hex
        - API_METRANG=https://backend.wmbus.testvps.acrios.com/parser/metraNG/hex
        - API_OLD_METRA=https://backend.wmbus.testvps.acrios.com/parser/metra/hex
        - BEARER_TOKEN=******** #REDACTED
        - MONGO_URI=mongodb://mongo:27017/
        - MONGO_DB_NAME=meter_database
        - MONGO_DB_VAR_METER_ID=meter_id
        - MONGO_DB_VAR_DRIVER=driver
        - MONGO_DB_VAR_AES_KEY=aes_key
        - MONGO_DB_VAR_IMPORT_TIMESTAMP_NAME=import_timestamp
        - IMPORT_FOLDER=/app/import
        volumes:
        - ./import:/app/import
        depends_on:
        - mongo
        - wmbusmeters
        networks:
        - mqtt-net

    networks:
    mqtt-net:
        driver: bridge

    Example with the parameters from the Public Mosquitto Broker.

    HiveMQ example (serverless)
    version: '3'
    services:
    mongo:
        image: mongo:4.4.18
        container_name: mongo_db
        ports:
        - "27017:27017"
        volumes:
        - ./db:/data/db
        networks:
        - mqtt-net

    wmbusmeters:
        build:
        context: ./wmbusmeters
        dockerfile: ./Dockerfile
        container_name: wmbusmeters_container
        networks:
        - mqtt-net

    mqtt-parser:
        build: .
        container_name: mqtt_parser
        environment:
        - PYTHONUNBUFFERED=1
        - BROKER=144ee8a9a2294bb99a8a97f3b4eb2ad3.s1.eu.hivemq.cloud
        - PORT=8883
        - USERNAME=acrios
        - PASSWORD=******** #REDACTED
        - TOPIC_PREFIX=acrios/+/meter/+/raw
        - ENABLE_INTERNAL_WMBUSMETERS_PARSER=true
        - API_WMBUSMETERS=https://backend.wmbus.testvps.acrios.com/parser/wmbusmeters/hex
        - API_METRANG=https://backend.wmbus.testvps.acrios.com/parser/metraNG/hex
        - API_OLD_METRA=https://backend.wmbus.testvps.acrios.com/parser/metra/hex
        - BEARER_TOKEN=******** #REDACTED
        - MONGO_URI=mongodb://mongo:27017/
        - MONGO_DB_NAME=meter_database
        - MONGO_DB_VAR_METER_ID=meter_id
        - MONGO_DB_VAR_DRIVER=driver
        - MONGO_DB_VAR_AES_KEY=aes_key
        - MONGO_DB_VAR_IMPORT_TIMESTAMP_NAME=import_timestamp
        - IMPORT_FOLDER=/app/import
        volumes:
        - ./import:/app/import
        depends_on:
        - mongo
        - wmbusmeters
        networks:
        - mqtt-net

    networks:
    mqtt-net:
        driver: bridge

    Example with the parameters from the HiveMQ Cloud Cluster that we have created in integration manual.

  3. Run the Docker container:

    docker-compose up -d

Parsed Messages​

  • Messages before parsing:
    em_p1
  • Messages after parsing:
    em_p2

    In this example AES keys weren't imported, so some messages are not decrypted.

HiveMQ example (serverless)
  • Messages before parsing:
    hm_p1
  • Messages after parsing:
    hm_p2

Node-RED MQTT Parser Flow​

The Node-RED flow is designed to subscribe to acrios/+/meter/+/raw topic, parse incoming messages, and publish processed data to the acrios/+/meter/+/parsed topic. The flow is designed to handle and parse messages from multiple devices and meters, making it easy to integrate with downstream applications.

  • Easy to use: Node-RED is a visual programming tool that makes it easy to create flows.
  • Configurable: Uses MQTT nodes to subscribe and publish messages, and function nodes to parse and process data.
caution

The Node-RED flow uses ACRIOS Systems' backend API to parse the messages. The Metra parser API is not publicly available! Contact ACRIOS Systems to get authorization for the API.

  1. Install Node-RED on your computer or server. You can find the installation instructions here.
  2. Open Node-RED in your browser.
    nodered1
  3. Click on the hamburger menu in the top right corner and select Manage palette.
    nodered2
  4. Click on the Install tab and search for node-red-contrib-http-request and node-red-contrib-mqtt-broker nodes. Click on the Install button to install the nodes.
    1. nodered3
    2. nodered4
  5. Installed nodes will appear in the Nodes tab. Close the Manage palette window.
    nodered5
  6. Click on the hamburger menu in the top right corner and select Import.
    nodered6
  7. Select flow file or copy the flow code, than click on Import.
    nodered7
    Show flow code or Download flow
  8. The imported flow will appear in the Node-RED workspace.
    nodered8
  9. Click on Subscribe to Raw meter data node and edit Server
    nodered9
  10. Modify setting for your MQTT broker, than navigate to Security tab.
    nodered10

    Example with the parameters from the HiveMQ Cloud Cluster that we have created.

  11. Fill in credentials and click on Update button.
    nodered11

    Example with the parameters from the HiveMQ Cloud Cluster that we have created.

  12. Click on Prepare data for parsing node and fill in authorization header, than click on Done button.
    nodered12
  13. Deploy the flow by clicking on the Deploy button in the top right corner.
    nodered13
Last updated on by gasper